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Final Rejection 
Response to Amendment 

1. It is hereby acknowledged that the following papers have been received and 
placed on record in the file: Amendment as received on 7-31-2006. 

Claims 1-46 have been examined. 
Status of Claims: 

2. Claims 17-23, 25-29, 31-33, 35-37, 39, 40, and 42-46 rejected under 35 
U.S.C. 102(e) as being anticipated by Filo etal., patent #6,215,498, hereinafter Filo. 

3. Claims 24, 30, and 34 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Filo et al., patent #6,215,498, hereinafter Filo and Nitta, patent 
#5,347,306. 

4. Claims 38 and 41 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Filo et al., patent #6,215,498, hereinafter Filo and Sun et al., patent #6,501,740, 
hereinafter Sun. 

5. Claims 1-16 have been cancelled by the applicant. 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

6. Claims 17 s 20 ; 35, 39 ; 40, 43, and 45 are rejected under 35 U.S.C. 112, first 
paragraph, as failing to comply with the enablement requirement. The claim(s) contains 
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subject matter which was not described in the specification in such a way as to enable 
one skilled in the art to which it pertains, or with which it is most nearly connected, to 
make and/or use the invention. Specifically, support has not bee pointed toward nor 
could any be found for "automatically" changing the user's mode of communication so 
as to accommodate the detected impairment". 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

8. Claims 17-23, 25-29, 31-33, 35-37, 39, 40, and 42-46 rejected under 35 
U.S.C. 102(e) as being anticipated by Filo etal., patent #6,215,498, hereinafter Filo. 

9. With regard to claims 17, 20, 35, 39, 40, 43, and 45, which teaches a system 
comprising: a mobile device for communication between a user of the mobile device 
and at least one other party, Filo teaches, in column 7, lines 9-15, and lines 25-36, a 
wearable personal computer that transmits changes in the users voice and positional 
data via a network to the personal computers of all other users in attendance of a virtual 
environment. With regard to the claims, which further teach a mobile user interface for 
connection with the mobile device including input means for providing the at least one 
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other party with a representation of the user, Filo teaches, in column 3, lines 1-23, users 
being represented by avatars on the display device of users computers, the avatar's 
representation is based on input from position trackers connected to the user. With 
regard to the claims, which further teach output means for providing the user with a 
representation of the at least one other party, Filo teaches, in column 3, lines 4-9, a 
display device showing a representation of avatars (users) in the environment. With 
regard to the claims, which further teach detecting means for detecting an 
environmental or physical attribute affecting the communication by the user, Filo 
teaches, in column 3, lines 9-15, the avatar's representation is based on input from 
position trackers connected to the user. With regard to the claims, which further teach 
processing means for processing data indicating the environmental or physical attribute, 
Filo teaches, in column 7, lines 9-15, a processing of the graphics in correspondence 
with manipulation of the avatars. With regard to the claims, which further teach means 
for configuring or modifying the representation of the at least one other party to the user 
in dependence on the detected environmental or physical attribute, Filo teaches, in 
column 3, lines 1-23, updating the representation of the avatar depending on the 
detected visual (head and hand movement) changes. With regard to the claims, which 
further teach means to configure or modify the representation of the user to the at least 
one other party in dependence on the detected environmental or physical attribute, Filo 
teaches, in column 3, line 1 through column 4, line 26, several users, each being 
represented by an avatar in the virtual environment, affecting changes in the other users 
display via physical movement. With regard to the claims, which further teach a 



Application/Control Number: 10/088,346 Page 5 

Art Unit: 2173 

computer for connection with the mobile device and/or the mobile user interface 
including means for defining a virtual meeting space for communications between the 
user and the at least one other party, Filo teaches, in column 8, lines 45-51 , the virtual 
environment including maps, documents, presentation materials, etc., and further 
teaches, in column 8, line 58 through column 9, line 5, avatars being moved into virtual 
rooms for meetings. With regard to the claims, which further teach means for 
controlling the representation of the user and the at least one other party to each other 
in dependence on the detected environmental or physical attribute, Filo teaches, in 
column 9, lines 14-39, avatars being presented to other users being manipulated by 
hand movements being detected through hand trackers. With regard to the claims 
further teaching detecting an environmental or physical attribute impairing the ability of 
the user to continue communication in a currently occurring mode and automatically 
changing the user's mode of communication so as to accommodate the detected 
impairment, Filo teaches, in column 10, line 51 through column 11, line 11, the system 
detecting a person from outside the environment attempting to connect with a user in a 
private conversation, to accommodate the user enters into a private phone call mode 
which disables the audio links to the VCP, and displays a phone icon next to the avatar 
to the other avatars. 

10. With regard to claim 18, which teaches the representation of the user comprising 
a moving visual image, Filo teaches, in column 9, lines 35-39, the user being 
represented by an avatar the moves through a series of articulated joints in response to 
changes in motion detected by the user's hand trackers. 
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1 1 . With regard to claim 1 9, which teaches the representation of the user comprising 
an avatar, Filo teaches, in column 9, lines 35-39, the user being represented by an 
avatar the moves through a series of articulated joints in response to changes in motion 
detected by the user's hand trackers. 

12. With regard to claim 21 , which teaches the output means comprises a first output 
mode and a second output mode, and the representation of the at least one other party 
is modified by switching the output means form the first output modes to the second 
output mode, Filo teaches, in column 13, lines 5-31, and in column 3, line 1 through 
column 4, line 26, the user being in one of two output modes, a seated user (being 
represented by seated avatar) and a standing presenter (whose movements jesters are 
recognized for output to the others in the virtual environment). 

13. With regard to claim 22, which teaches the first output mode comprising a video 
output, and the second output mode comprises an audio output, Filo further teaches, in 
column 13, lines 43-55 and column 10, lines 51-54, an output mode, where a non- 
immersed user is able to provide audio and/or video output to the session. 

14. With regard to claim 23, further teaching means to provide the user with 
information alerts related to an automatically detected environmental or physical 
condition, Filo teaches, column 13, lines 43-55 and column 10, lines 51-54, an output 
mode, where a non-immersed user is able to provide live audio and/or video to the 
session. Filo further teaches, in column 10, line 51 through column 1 1 , line 1 1 , the 
system detecting a person from outside the environment attempting to connect with a 
user in a private conversation, to accommodate the user enters into a private phone call 
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mode which disables the audio links to the VCP, and displays a phone icon next to the 
avatar to the other avatars. 

15. With regard to claim 25, further teaching the environmental or physical attribute is 
a change in an acceleration or a force associated with a user, Filo teaches, in column 9, 
lines 35-39, the user being represented by an avatar the moves through a series of 
articulated joints in response to changes in motion detected by the user's hand trackers, 
where the movement from a stationary state is an acceleration of force. 

16. With regard to claim 26, further teaching the environmental or physical attribute is 
a location, Filo teaches, in column 8, line 58 through column 9, line 5 and lines 40-60, 
avatars being moved into virtual rooms for meetings. 

17. With regard to claim 27, further teaching the environmental or physical attribute is 
an attribute is a direction of changing user location, Filo teaches, in column 3, lines 15- 
19, correlating changes in hand an head movement, where this correlation need 
coincide with the direction. 

18. With regard to claim 28, further teaching the environmental or physical attribute is 
a change in velocity, Filo teaches, in column 9, lines 35-39, the user being represented 
by an avatar the moves through a series of articulated joints in response to changes in 
motion detected by the user's hand trackers, where the movement is a velocity. 

19. With regard to claim 29, further teaching the environmental or physical attribute is 
a movement of a user, Filo teaches, in column 9, lines 35-39, the user being 
represented by an avatar the moves through a series of articulated joints in response to 
changes in motion detected by the user's hand trackers, where the movement is a 
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velocity; and in column 8, line 58 through column 9, line 5 and lines 40-60, avatars 
being moved into virtual rooms for meetings. 

20. With regard to claim 31 , further teaching the environmental or physical attribute is 
privacy level of the user, Filo teaches, in column 9, lines 14-22, the user being able to 
view number of people in the virtual room, showing him how private the conversation 
will be. 

21 . With regard to claim 32, further teaching processing means for processing data 
indicative of the environmental or physical attribute, Filo teaches, in column 7, lines 9- 
15, a processing of the graphics in correspondence with manipulation of the avatars. 

22. With regard to claim 33, which teaches the processing means utilizing a 
classification or analysis algorithm, Filo teaches, in column 3, lines 4-19, analyzing hand 
and head movements and mapping them to the virtual avatar on the screen. 

23. With regard to claim 36, further teaching a store for storing predetermined 
information corresponding to the detected environmental or physical attribute, Filo 
teaches, in column 3, lines 8-25, a database for storing computer information and 
graphics supporting the appearance of the avatars that populated the virtual work 
environment. 

24. With regard to claims 37, 42, 44, and 46, which teach communication between 
the user of the mobile device and more than one other party, Filo teaches, in column 7, 
lines 9-15, and lines 25-36, a wearable personal computer that transmits changes in the 
users voice and positional data via a network to the persona! computers of all other 
users in attendance of a virtual environment. 
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Claim Rejections - 35 (JSC § 103 

25. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

26. Claims 24, 30, and 34 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Filo et al., patent #6,215,498, hereinafter Filo and Nitta, patent 
#5,347,306. 

27. With regard to claim 24, Filo teaches, in column 3, line 1 through column 4, line 
26, Several users, each being represented by an avatar in the virtual environment, 
affecting changes in the other users display via physical movement, but doesn't 
specifically teach the environmental or physical attribute is an ambient sound level. Nitta 
teaches, in column 3, lines 1-15, a virtual meeting place with participants being 
represented as avatars, similar to that of Filo, but further teaches, in column 3, lines 25- 
39, column 7, lines 20-26, and column 6, lines 41-50, sound being steered toward the 
individual determined to be speaking, where ambient noise is picked up and determined 
to be relevant or extraneous. It would have been obvious to one of ordinary skill in the 
art, having the teachings of Filo and Nitta before him at the time the invention was made 
to modify virtual presentation system of Filo to include ambient noise and focusing the 
audio to minimize the ambient sound volume. One would have been motivated to make 
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such a combination because ambient noise is a part of every conversation and need be 
recognized as such. 

28. With regard to claim 30, Filo teaches, in column 3, line 1 through column 4, line 
26, Several users, each being represented by an avatar in the virtual environment, 
affecting changes in the other users display via physical movement, but doesn't 
specifically teach the environmental or physical attribute is a posture of the user. Nitta 
teaches, in column 3, lines 1-15, a virtual meeting place with participants being 
represented as avatars, similar to that of Filo, but further teaches, in column 3, lines 1 2- 
16, the users posture being a physical attribute. It would have been obvious to one of 
ordinary skill in the art, having the teachings of Filo and Nitta before him at the time the 
invention was made to modify virtual presentation system of Filo to include posture 
changes. One would have been motivated to make such a combination because this 
provides a user with a better representation of the other users in the system. 

29. With regard to claim 34, Filo teaches, in column 3, line 1 through column 4, line 
26, Several users, each being represented by an avatar in the virtual environment, 
affecting changes in the other users display via physical movement, but doesn't 
specifically teach the processing means utilizing a Hidden Markov Model. Nitta 
teaches, in column 3, lines 1-15, a virtual meeting place with participants being 
represented as avatars, similar to that of Filo, but further teaches, in column 3, lines 35- 
40 and column 6, lines 41-50, the processing by knowing the a user exists without 
actually knowing the users exact position, but rather estimating a position. This process 
obviously could be implemented by or is actually being implemented with a Hidden 
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Markov Model. It would have been obvious to one of ordinary skill in the art, having the 
teachings of Filo and Nitta before him at the time the invention was made to modify the 
virtual meeting system of Filo to include this hidden state system of Nitta. One would 
have been motivated to make such a combination because this allows physical changes 
without the certainty of user definition. 

30. Claims 38 and 41 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Filo et al., patent #6,21 5,498, hereinafter Filo and Sun et al., patent #6,501 ,740, 
hereinafter Sun. 

31. With regard to claims 38 and 41, Filo teaches, in column 3, line 1 through column 
4, line 26, Several users, each being represented by an avatar in the virtual 
environment, affecting changes in the other users display via physical movement and 
Filo further teaches, in column 10, line 51 through column 11, line 11, the system 
detecting a person from outside the environment attempting to connect with a user in a 
private conversation, to accommodate the user enters into a private phone call mode 
which disables the audio links to the VCP, and displays a phone icon next to the avatar 
to the other avatars, but doesn't specifically teach detecting a level of service in the 
connection between the computers, and modifying the representation of the user to the 
at least one other party in dependent on the detected quality of service. Sun teaches a 
teleconference system where conferees can be presented with virtual reality animations 
(see column 5, lines 50-63) ? but further teaches the system recognizing the type of 
connection of each conferee connected to the teleconference, and adapting the signal 
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sent to a conferee based on the connection, for example only the audio portion of a 
signal from a video conference is sent to a conferee connected by a conventional 
telephone (see column 5, line 50 through column 6, line 34). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Filo to include the 
exclusion of the video portion to a participant that doesn't have video capabilities and to 
display the limited connectivity user with an icon as was done with the phone icon of 
Filo. One would have been motivated to make such a combination because a 
participant without video capabilities would be unable to view the presenter anyway. 

Response to Arguments 

32. The arguments filed on 7-31-2006 have been fully considered but they are not 
persuasive. Reasons set forth below. 

33. The applicant argues that Filo does not teach any means for detecting 
environmental or physical attributes that impair the ability o a user to continue 
communication in a currently occurring mode-coupled with means for automatically 
changing the user's mode of communication so as to accommodate the detected 
impairment. 

34. In response the examiner respectfully submits that Filo teaches, in column 10, 
line 51 through column 11, line 11, the system detecting a person from outside the 
environment attempting to connect with a user in a private conversation, to 
accommodate the user enters into a private phone call mode which disables the audio 
links to the VCP, and displays a phone icon next to the avatar to the other avatars. 



Application/Control Number: 10/088,346 



Art Unit: 2173 



Page 13 



35. The applicant argues that neither Filo nor Nitta teach employing a Hidden Markov 
Model approach to detect environmental of physical attributes. 

36. In response the examiner respectfully submits that using the applicants supplied 
definition, Nitta teaches determining a physical location of a user using an 
approximation based on estimating where a talking user is using recognized sound, 
attempting to ignore extraneous noise, and further relying on lip movements (see 
column 3, lines 35-40 and column 6, lines 41-50). Here the "state" or "hidden state" is 
the users location which is determined based on a pattern recognition of the source of 
sound and lip movement. This process obviously could be implemented by or is 
actually being implemented with a Hidden Markov Model. 

37. The applicant argues that Sun does not teach that "a representation of the user 
being provided to the other conference parties would be modified in dependence upon 
the detected "quality of service" of the connection." 

38. In response the examiner respectfully submits that Filo further teaches, in column 
10, line 51 through column 11, line 11, the system detecting a person from outside the 
environment attempting to connect with a user in a private conversation, to 
accommodate the user enters into a private phone call mode which disables the audio 
links to the VCP, and displays a phone icon next to the avatar to the other avatars. Sun 
teaches the system recognizing the type of connection of each conferee connected to 
the teleconference, and adapting the signal sent to a conferee based on the connection, 
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for example only the audio portion of a signal from a video conference is sent to a 
conferee connected by a conventional telephone (see column 5, line 50 through column 
6, line 34). It would have been obvious to one of ordinary skill in the art, having the 
teachings of Filo to include the exclusion of the video portion to a participant that 
doesn't have video capabilities and to display the limited connectivity user with an icon 
as was done with the phone icon of Filo. One would have been motivated to make such 
a combination because a participant without video capabilities would be unable to view 
the presenter anyway. 

Conclusion 

39. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

40. A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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41 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis G. Bonshock whose telephone number is (571) 
272-4047. The examiner can normally be reached on Monday - Friday, 6:30 a.m. - 4:00 
p.m. 

42. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid can be reached on (571) 272-4063. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

43. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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